Motion estimation is an effective inter-frame coding technique to exploit temporal redundancy in video sequences. Motion-compensated inter-frame coding has been widely used in various international video coding standards The motion estimation adopted in various coding standards is often a block-based technique, where motion information such as coding mode and motion vector is determined for each macroblock or similar block configuration. In addition, intra-coding is also adaptively applied, where the picture is processed without reference to any other picture. The inter-predicted or intra-predicted residues are usually further processed by transformation, quantization, and entropy coding to generate compressed video bitstream. During the encoding process, coding artifacts are introduced, particularly in the quantization process. In order to alleviate the coding artifacts, additional processing has been applied to reconstructed video to enhance picture quality in newer coding systems. The additional processing is often configured in an in-loop operation so that the encoder and decoder may derive the same reference pictures to achieve improved system performance.
FIG. 1A illustrates an exemplary system block diagram for an video encoder based on High Efficiency Vide Coding (HEVC) using adaptive Inter/Intra prediction. For Inter-prediction, Motion Estimation (ME)/Motion Compensation (MC) 112 is used to provide prediction data based on video data from other picture or pictures. Switch 114 selects Intra Prediction 110 or Inter-prediction data and the selected prediction data is supplied to Adder 116 to form prediction errors, also called residues. The prediction error is then processed by Transformation (T) 118 followed by Quantization (Q) 120. The transformed and quantized residues are then coded by Entropy Encoder 122 to form a video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packed with side information such as motion, mode, and other information associated with the image area. The side information may also be subject to entropy coding to reduce required bandwidth. Accordingly, the data associated with the side information are provided to Entropy Encoder 122 as shown in FIG. 1A. When an Inter-prediction mode is used, a reference picture or pictures have to be reconstructed at the encoder end as well. Consequently, the transformed and quantized residues are processed by Inverse Quantization (IQ) 124 and Inverse Transformation (IT) 126 to recover the residues. The residues are then added back to prediction data 136 at Reconstruction (REC) 128 to reconstruct video data. The reconstructed video data may be stored in Reference Picture Buffer 134 and used for prediction of other frames.
As shown in FIG. 1A, incoming video data undergoes a series of processing in the encoding system. The reconstructed video data from REC 128 may be subject to various impairments due to a series of processing. Accordingly, various in-loop processing is applied to the reconstructed video data before the reconstructed video data are stored in the Reference Picture Buffer 134 in order to improve video quality. In the High Efficiency Video Coding (HEVC) standard being developed, Deblocking Filter (DF) 130 and Sample Adaptive Offset (SAO) 131 have been developed to enhance picture quality. The in-loop filter information may have to be incorporated in the bitstream so that a decoder can properly recover the required information. Therefore, in-loop filter information from SAO is provided to Entropy Encoder 122 for incorporation into the bitstream. In FIG. 1A, DF 130 is applied to the reconstructed video first; SAO 131 is then applied to DF-processed video. However, the processing order among DF and SAO can be re-arranged.
A corresponding decoder for the encoder of FIG. 1A is shown in FIG. 1B. The video bitstream is decoded by Video Decoder 142 to recover the transformed and quantized residues, SAO information and other system information. At the decoder side, only Motion Compensation (MC) 113 is performed instead of ME/MC. The decoding process is similar to the reconstruction loop at the encoder side. The recovered transformed and quantized residues, SAO information and other system information are used to reconstruct the video data. The reconstructed video is further processed by DF 130 and SAO 131 to produce the final enhanced decoded video.
In the High Efficiency Video Coding (HEVC) system, the fixed-size macroblock of H.264/AVC is replaced by a flexible block, named coding unit (CU). Pixels in the CU share the same coding parameters to improve coding efficiency. A CU may begin with a largest CU (LCU, also referred as CTU, coded tree unit in HEVC). In addition to the concept of coding unit, the concept of prediction unit (PU) is also introduced in HEVC. Once the splitting of CU hierarchical tree is done, each leaf CU is further split into prediction units (PUs) according to prediction type and PU partition. The Inter/Intra prediction process in HEVC is applied to the PU basis. For each 2N×2N leaf CU, a partition size is selected to partition the CU. A 2N×2N PU may be partitioned into 2N×2N, 2N×N, or N×2N PU when Inter mode is selected. When a 2N×2N PU is Intra coded, the PU may be partitioned into either one 2N×2N or four N×N.
While non-overlapped motion prediction blocks are most used in HEVC practice, there are also proposals for overlapped motion compensation presented during HEVC standard development. Overlapped Block Motion Compensation (OBMC) is a technical proposed during the HEVC standard development. OBMC utilizes Linear Minimum Mean Squared Error (LMMSE) technique to estimate a pixel intensity value based on motion-compensated signals derived from neighboring block motion vectors (MVs). From estimation-theoretic perspective, these MVs are regarded as different plausible hypotheses for its true motion, and to maximize coding efficiency, their weights should minimize the mean squared prediction error subject to the unit-gain constraint.
An OBMC proposal during HEVC development is disclosed in JCTVC-C251 (Chen, et al, “Overlapped block motion compensation in TMuC”, in Joint Collaborative Team on Video Coding (JCT-VC), of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11 3rd Meeting: Guangzhou, CN, 7-15 Oct. 2010, Document: JCTVC-C251), where OBMC is applied to geometry partition. In geometry partition, it is very likely that a transform block contains pixels belonging to different partitions since two different motion vectors are used for motion compensation. Therefore, the pixels at the partition boundary may have large discontinuities that can produce visual artifacts similar to blockiness. This in turn decreases the coding efficiency since the signal energy in the transform domain will spread wider toward high frequencies. Let the two regions created by a geometry partition be denoted as region 1 and region 2. The zig-zag line segments (210) indicate the partition line for region 1 and region 2. A pixel from region 1 (2) is defined to be a boundary pixel if any of its four connected neighbors (left, top, right, and bottom) belongs to region 2 (1). FIG. 2 illustrates an example, where pixels corresponding to the boundary of region 1 are indicated by pattern 1 and pixels corresponding to the boundary of region 2 are indicated by pattern 2. If a pixel is a boundary pixel (indicated by pattern 1 or 2), the motion compensation is performed using a weighted sum of the motion predictions from the two motion vectors. The weights are ¾ for the prediction using the motion vector of the region containing the boundary pixel and ¼ for the prediction using the motion vector of the other region. In other words, the pixel at the boundary is derived from the weighted sum of two predictors corresponding to two different motion vectors. The overlapping boundaries improve the visual quality of the reconstructed video while providing BD-rate gain.
Another OBMC proposal during the HEVC standard development is disclosed in JCTVC-F299 (Guo, et al, “CE2: Overlapped Block Motion Compensation for 2N×N and N×2N Motion Partitions”, in Joint Collaborative Team on Video Coding (JCT-VC), of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 6th Meeting: Torino, 14-22 Jul. 2011, Document: JCTVC-F299), where OBMC is applied to symmetrical motion partitions. If a coding unit (CU) is partitioned into two 2N×N or N×2N partition units (PUs), OBMC is applied to the horizontal boundary of the two 2N×N prediction blocks, and the vertical boundary of the two N×2N prediction blocks. Since those partitions may have different motion vectors, the pixels at partition boundary (i.e., PU boundaries) may have large discontinuities, which may generate visual artifacts and also reduce the coding efficiency. In JCTVC-F299, OBMC is introduced to smooth the boundaries of motion partition.
FIG. 3 illustrates exemplary OBMC for 2N×N (FIG. 3A) and N×2N blocks (FIG. 3B). The pixels in the shaded area belong to Partition 0 and the pixels in the clear area belong to Partition 1. The overlapped region in the luma component is defined as 2 rows (or columns) of pixels on each side of the horizontal (or vertical) PU boundary. For pixels that are 1 row (or column) apart from the partition boundary, i.e., pixels labeled as A in FIG. 3, OBMC weighting factors are (¾, ¼). For pixels that are 2 rows (columns) away from the partition boundary, i.e., pixels labeled as B in FIG. 3, OBMC weighting factors are (⅞, ⅛). For chroma components, the overlapped region is defined as 1 row (or column) of pixels on each side of the horizontal (or vertical) PU boundary, and the weighting factors are (¾, ¼).